Genuine bible-thumpers will ignore any and all evidence to the contrary.
And discrediting the Noachian flood is more straightforward than this thread about the Tower of Babel. But in either case, the bible doesn't talk about the magic where the magic would need to be. Genesis talks more about the details of the size of the ark, the fact that god closed the door, the boring details of sending out the birds, and then finishing off with the account with promptly slaughtering some of the surviving animals and burning them to make god happy. Hmmm, that's a lot of fuss about rather odd details when the story could be talking about the amazing journey of the kangaroos to and from the ark, or where the fresh water came from, or how they fit all the required food and got rid of the poop... or was there a miraculous pooping of manna by all the ark-dwellers.
If there was magic population growth, why didn't the bible enumerate that anybody had a lot of kids. It doesn't. Rather, the bible talks about the magic ages, that these people lived to 600 years, 430 years, 209 years. Later in Chap 11, it uses the "meanwhile he became father to sons and daughters". If the numbers of kids was a miracle, why didn't it boast about that instead of their ages? ... Well, that's kinda how legends turn out.
As far as population growth, the numbers can be crunched to come up with rapid increases. But really look at the account, in context, in Gen. 11:1-9. The chapter before, names all the sons in the geneology, only about a page worth, and concludes with stating to the effect, "these were the ones that spread about to make the nations" (omitting nations like China that have a recorded history predating the flood). The account of the tower reads like a complete insertion between two sections of geneology. Besides giving no numbers, it names no human names, just naming the city Babel... however, the previous chapter already said that Nimrod established 3 other cities as well. That would mean the tower account would have been earlier in the life of Nimrod... which would be a very small population. The tower account suggests a very early timing, setting it when all mankind was one nomadic group wandering west, then settling at the Shinar valley. They start building a tower. Jehovah freaks out and sends them travelling in all directions when there's only about 100 families. Don't forget, God loves incest. So split up this population so brothers and sisters have to marry each other again.
For Genesis to supposedly be the inspired word of god, people have to fabricate a lot more stories outside the text to try to fill the huge gaps left by these disjointed legends.